Skip to content

feat(query): narrow joined columns proven not NULL by where - #103

Merged
Makisuo merged 6 commits into
MapleTechLabs:mainfrom
artile:feat/predicate-facts
Nov 2, 2026
Merged

Makisuo merged 6 commits into
MapleTechLabs:mainfrom
artile:feat/predicate-facts

Conversation

@artile

@artile artile commented Oct 25, 2026 •

Copy link
Copy Markdown
Contributor

A LEFT JOIN types its columns as T | null. A where that proves a column present (isNotNull(), eq(…)) did not change that, so app code added ! or extra checks after the query.

Closes #67.
Part of #52.

Changes

  • Left-join nullability was already there (NullableColumnDefs, NullOr codecs). This PR adds the narrowing on top of it.
  • Condition gets a second type parameter, F, for not-NULL facts. It defaults to {}, so existing code does not change. A condition can drop a fact on assignment but cannot gain one.
  • On a joined column, isNotNull() and eq(…) return a condition with a fact (NULL = x is never true). FROM columns give no fact.
  • .and(…), and(…), the items of a where list and a second where keep the facts. or, .or, not, when, whenTrue, an item typed | undefined and a list of unknown length give none.
  • where returns NarrowJoins<Joins, Facts>: a proven column loses | null, and a Nullable column loses its own NULL too. With no facts, Joins stays the same type.
  • Runtime: select records the where conditions that exist when it is called (selectWhereFn). compile reads their facts (src/ch/facts.ts, a WeakMap like the tenant evidence) and gives the proven joined columns their non-null codec. So the row type and the derived row schema agree, and a NULL in a narrowed column fails to decode. This also works for leftJoinQuery and for a fromQuery over a narrowed query.
  • ColumnRef gets a phantom _columnType. Without it, the new Source parameter made the compiler compare refs member by member, and strRef.get("k") on a non-Map column stopped being a type error.
  • Tests: src/ch/predicate-facts.test-d.ts (types), src/ch/predicate-facts.test.ts (row schema, ClickHouse and Postgres), a live case left-join-not-null in tests/core-cases.ts, and a declaration-emit export in tests/package-consumer-declarations.mts.
  • Docs: docs/joins-and-subqueries.md (new section, backed by a docs example test), docs/reference.md. Changeset: minor.

Left out

  • Order: only a select after the where is narrowed. A select made before it keeps | null, because its row type is fixed when you call it. Narrowing an earlier select needs the output to remember which column each field came from. That is a larger change.
  • FROM-table columns declared Nullable: no narrowing yet. This keeps the Cols type unchanged in this PR. The same mechanism can cover them later.
  • in_(…) narrowing, and narrowing a literal union by eq / in_ (the issue's third "done when" item). This PR only removes null.
  • OR / NOT reasoning (for example not(isNull()), or the same fact on both sides of an or). These prove nothing in this PR, which is sound but less precise.
  • The runtime schema can be stricter than the type in one case: a fact inside an item typed | undefined, or inside a list of unknown length, that renders at run time. The type keeps | null there, and the decoder refuses a NULL that SQL cannot return anyway.
  • ClickHouse without join_use_nulls=1 fills a missing join row with the column default, not NULL. isNotNull() keeps that row. The narrowed type is still correct, because the value is not NULL. The live case shows both settings.
  • Nothing needs SQLite, MySQL or D1.

Validation

  • bun run build: pass.
  • bun run typecheck: pass (src and tests).
  • bunx vitest run --testTimeout=60000 --hookTimeout=60000 with a live ClickHouse 26.8.2.7 (EFFECT_ORM_CLICKHOUSE_URL / _USER / _PASSWORD set): 61 files passed, 1047 tests passed, 0 skipped files.
  • node scripts/check-doc-citations.mjs, node scripts/check-doc-examples.mjs, node scripts/check-exports-documented.mjs: pass.
  • bun scripts/check-package.ts: pass (declaration emit includes a narrowed query).
  • Upstream CI for a fork PR waits for a maintainer to approve the run.

View with [code]smith Autofix with [code]smith
Need help on this PR? Tag @codesmith-bot with what you need.

@coderabbitai

coderabbitai Bot commented Oct 25, 2026 •

Copy link
Copy Markdown

Warning

Review limit reached

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Next included review available in 37 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used the included review currently available.

Learn how review limits work.

Review configuration:

Run configuration
  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: defaults
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 7376e98b-6c8e-40e8-a834-388148d241d4
Commits

Reviewing files that changed from the base of the PR and between b57faf1 and 04ce9c5.

Files ignored due to path filters: 1
  • tests/__snapshots__/core-sql.test.ts.snap is excluded by !**/*.snap
Files selected for processing: 12
  • .changeset/predicate-facts-joins.md
  • docs/joins-and-subqueries.md
  • docs/reference.md
  • src/ch/compile.ts
  • src/ch/expr.ts
  • src/ch/facts.ts
  • src/ch/predicate-facts.test-d.ts
  • src/ch/predicate-facts.test.ts
  • src/ch/query.ts
  • src/docs-examples.test.ts
  • tests/core-cases.ts
  • tests/package-consumer-declarations.mts
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

Makisuo and others added 2 commits November 2, 2026 16:44
Conflicts: compile.ts imports (schemaOf from main, withoutNull from this
branch), the consumer-declarations test (both sides' exports kept) and the
core SQL snapshots (main's, with left-join-not-null recorded for every
dialect, sqlite and d1 included).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The narrowing landed before the /sqlite entry existed. Check that its
nullable type narrows in the row type and in the row schema (also under
d1Dialect), and that the /postgres entry's own types narrow on eq().

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Makisuo

Makisuo commented Nov 2, 2026

Copy link
Copy Markdown
Contributor

Merge-readiness pass (maintainer side):

  • Merged current main. Three conflicts:
    • src/ch/compile.ts imports: kept schemaOf from main and withoutNull / notNullFactsOf from this branch.
    • tests/package-consumer-declarations.mts: kept both sides' exports (the narrowed-query declarations and main's SQLite ones).
    • Core SQL snapshot: took main's, then recorded left-join-not-null for all four dialects. That case now also runs live on SQLite and D1.
  • Checked the narrowing against the new /sqlite entry. Its nullable has the same _tag: "Nullable" / element shape, so nonNullColumn already strips it. New tests:
    • Runtime: SQLite and D1 left joins narrow both in the row schema (a NULL fails to decode) and in the SQL.
    • Types: a SQLite nullable(text) joined column narrows to string and CompiledQuery carries it. The /postgres entry's own nullable narrows on eq().
  • Soundness check on eq proving not-NULL: eq(null) is refused by the builder (isNull() / isNotNull() only), so x = y with an eq fact can never match a NULL x. The changeset was already there.

Verified locally against ClickHouse 26.8:

  • bun run build, bun run typecheck.
  • bun run test: 74 files / 1448 tests, plus doc citations, the export catalog and doc examples.
  • Live tests/*.clickhouse.test.ts plus core.sqlite / core.postgres: 11 files / 290 tests.
  • vitest run tests/fuzz with FUZZ_SCALE=0.3: 269 tests.
  • bun scripts/check-package.ts: declaration emit and the CommonJS consumer.

All pass.

@Makisuo
Makisuo merged commit cf5a8f6 into MapleTechLabs:main Nov 2, 2026
1 of 3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Predicate facts: narrow row types from joins and WHERE

2 participants